Certain applications by their nature require vast amounts of data. For example, Wireless Sensor Networks (“WSN”) as described in U.S. Pat. No. 7,369,047 to Alan S. Broad et al. may collect vast amounts of data representing various physical properties (e.g., temperature or humidity) at different times. Multiple modules in the WSN sense the physical properties and relay sensor data wirelessly to a base station. The data collected at the base station may then be funneled into a database for storage. As the number of modules or the frequency of data collection increases, the amount of data collected by the WSN increases proportionally. Sometimes, the data may be collected over an extended period of time ranging from months to years, resulting in considerable amounts of data for storage and processing.
When a query request is received from a user, a search may be performed on the database to retrieve pertinent data sets. The query often requires binary searches across data entries stored in the database. Although it is preferable to return query results to the user instantaneously, the data access time increases as the amount of data stored in the database increases. The increased data access time makes it difficult to satisfy users' demand for real-time response. Especially if the data requests are received simultaneously from multiple users, the database may be burdened with numerous requests and fail to return the results in real-time.
The retrieved data sets may be processed and presented to the user using visual tools such as graphs or charts to help users identify any trends or correlation between parameters. If the number of retrieved data sets is large, the graphs or charts may become overpopulated with data. Further, if the retrieved data set exceeds the number of pixels allocated to the graphs or charts, only a subset of retrieved data sets may be displayed on the screen. To select the subset of retrieved data to be displayed, the retrieved data must be decimation or summarized to a smaller number appropriate for display. As more data sets are retrieved, more time is required to decimate or summarize the retrieved data sets. The accumulated retrieval time and processing time may result in a significant delay in presenting the graphs or charts to the user.